TWO ORBITS: 
WHEN IS ONE IN THE CLOSURE OF THE OTHER? 



VLADIMIR L. POPOV 
To V. A. Iskovskikh on the occasion of his 70th birthday 

Abstract. Let G be a connected linear algebraic group, let V he a finite 
dimensional algebraic G- module, and let Oi, O2 be two G-orbits in V. 
We describe a constructive way to find out whether or not Oi lies in the 
closure of 02- 



1. Introduction 

1.1. Fix an algebraically closed ground field k of arbitrary characteristic. 

Let G be a connected linear algebraic group and let be a finite dimensional 
algebraic G-module. Consider two points a and h and their G-orbits G ■ a 
and G • b. 

The following problem continually arises in algebraic transformation group 
theory and its applications: 

How can one find out whether or not the orbit G ■ a lies in , , 

(*) 

the closure in V of the orbit G ■ b? ^ ' 

(here and further topological terms are related to the Zarisky topology). 

Example 1.2. If the group G is reductive and a = 0, then Problem (*) means 
finding out whether or not the point b is unstable in the sense of Geometric In- 
variant Theory [M65]. A description of the cone of unstable points is provided 
by the Hilbert-Mumford theory. 

Example 1.3. Let G be a torus and let X{G) be the group of its characters 
in additive notation. For every A € X{G), g £ G, and v & V denote by 
and vx respectively the value of A at g and the projection of v to the A- 
wight subspace of the G-module V parallel to the sum of the other weight 
subspaces. Let suppv := {A G X{G) \ vx + 0}. Then by [PV72] Problem (*) 
means finding out whether or not the following conditions hold: (i) the cone 
generated by suppa in X{G) ®i M is a face of the cone generated by supp&, 
and (ii) there is an element g £ G such that g'^ax = bx for every A G suppa. 

Example 1.4. If the group G is unipotent, then every G-orbit is closed in V , 
see [R61]. Therefore Problem (*) means finding out whether or not the points 
a and b lie in one and the same G-orbit. 
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Example 1.5. Let char A; = 0. Assume that G is a simple group, V is its Lie 
algebra endowed with the adjoint action of G, and the elements a and b are 
nilpotent. If G is a classical group (i.e., of type A;, B;, C;, or D;), then the 
answer to (*) is given by the known rule formulated in terms of the sizes of 
Jordan blocks of the Jordan normal forms of a and b, sec, e.g., [CM93]. If the 
group G is exceptional (i.e., of type Eg, E7, Eg, F4, or G2), then this answer, 
obtained by means of ad hoc methods, is given by the explicit Hasse diagrams 
of the set of nilpotent orbits endowed with the Bruhat order, i.e., partially 
ordered according to the rule 

(as usual, bar means the closure in V), see [Spa82], [C85]. 

Example 1.6. Apart from the classical case of orbits of a Borel subgroup of 
a reductive group G on the generalized flag variety G/P, and also the case 
of Example 1.5, the Hasse diagrams of the sets of orbits endowed with the 
Bruhat order are found utilizing the ad hoc methods in some other special 
cases, see, e.g., [Kas90], [Pe04], [BHRZ99], [GHR07], [MWZ99], and Examples 
1.7 and 1.8 below. On the other hand, in a number of cases the orbits are 
classified, but the Hasse diagrams are not found: for instance, this is so for 
nilpotent 3-vectors of the n-dimensional spaces where n ^ 9, for 4-vectors of 
an 8-dimensional space, for spinors of the m-dimensional spaces where m ^ 14 
and 16, see the relevant references in [PV94]. 

Example 1.7. Let L be a finite dimensional vector space over k. Let G = 
GL(L) and F = L* (g) L* (g) L. Points of V are the structures of (not necessarily 
associative) fc-algebras on the vector space L. The algebras defined by the 
structures a and b are isomorphic if and only if G-a = G-b. In the language of 
the theory of algebras Problem (*) is formulated as follows: How can one find 
out whether or not the algebra defined by the structure a is a degeneration 
of the algebra defined by the structure b? In general case it is considered to 
be a difficult problem. There is a number of papers where a classification of 
degenerations in various special cases is obtained by means of ad hoc methods, 
see, e.g., [B05], [BS99], [See90], and survey [OVG94, Chap. 7]. 

Example 1.8. Consider the conjugation action of the group G = GLd{k) 
on Matrf (i(A;). Consider a finite dimensional associative /c-algebra A and a 
d-dimensional vector space L over k. If a basis in A and a basis in L are 
fixed, then the set of structures of left A-modules on L is naturally identified 
with a closed invariant subset Mod^ of the direct sum of dimj^yl copies of 
the G-module Matfi^dik)- Denote by Mx the A-module corresponding to a 
point X G Mod^. Then the ^-modules Ma and are isomorphic if and only 
if G ■ a = G ■ b, and in this theory the condition G • a C G • 6 is expressed 
by saying that Ma is a degeneration of M^. If A is the path algebra of a 
quiver obtained by fixing an orientation of the extended Dynkin graph of a 
root system of type A;, D;, Eg, E7, or Eg, then in [B95] a characterization of 
the degeneration relation in terms of ^-module structures of Ma and Mf, and 
an algorithm that finds out whether or not Ma is the degeneration of Mi, are 
obtained. 
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Example 1.9. In case of the natural action of the group GL„(/c) on the space 
of n-ary forms of degree d with the coefficients in k Problem (*) (under the 
name The orbit closure problem) is of fundamental importance in application 
of geometric invariant theory to complexity theory, see [MSOl]. 

1.10. In this paper we give a constructive solution to Problem (*): in Section 
2 we suggest an algorithm that provides an answer to (*) by means of a finite 
number of effectively feasible operations. Namely, we explicitly point out a 
finite system of linear equations in finitely many variables over the field k such 
that the inclusion G ■ a C G ■ b is equivalent to its inconsistency (the precise 
formulation is contained in Theorem 2.12). By Kronecker-Capelli theorem 
this reduces answering (*) to comparing ranks of two explicitly given matrices 
with the coefficients in k that can be executed constructively. Of course, the 
existence of a constructive solution to Problem (*) immediately leads to the 
problem of finding a most effective algorithm. But this is another problem 
that we do not consider here. 

1.11. In Section 3 we suggest another algorithm for finding an answer to 
Problem (*). It is less effective than the algorithm from Section 2, but it 
provides more information and concerns a more general problem. To wit, let 
L be a linear subvariety of V. We show how one can constructively find a 
finite system of polynomial functions qi, ... ,qm on V such that 

G^={xeV\qi{x) = ... = qm{x)=0}. (1) 

For L = b this provides the following constructive answer to Problem (*): 

G ■ a C. G ■ b = • • • = qm{o) = 0- 

Note that varieties of the form G ■ L are ubiquitous in algebraic transforma- 
tion group theory: apart from orbit closures, to them also belong irreducible 
components of Hilbert null-cones and, more generally, closures of Hesselink 
strata [Po03], closures of sheets [PV94], and closures of Jordan (a.k.a. decom- 
position) classes [TY05] . Also note that if a system of polynomials gi , . . . 
. . . ,qm satisfying (1) is given, modern commutative algebra provides algo- 
rithms to constructively find a system of generators of the ideal of all poly- 
nomials vanishing on G ■ L, see, e.g., [CL098, Chap. 4, §2]. In particular, this 
provides methods to constructively find generators of the ideal of polynomials 
vanishing on the closure of orbit. In some special cases (for instance, for nilpo- 
tent orbits of the adjoint action of the group SL„(A;) and for "rank varieties") 
such generators have been found, see [W89]. 

1.12. In essence, both algorithms are based on the possibility to rationally 
parametrize an open subset of G by means of a variety of the form 

A'-'^ := {(ei, . . . , Er+s) G A'-+^ | . . . ^ o}, r,seN 

(we denote by N the set of all nonnegative integers), more precisely, on the 
existence of a dominant morphism 



6: A^'^^G. 



(2) 
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1.13. As every normal quasiprojeetive variety endowed with an algebraic ac- 
tion of G can be equivariantly embedded in a projective space [PV94], in 
algebraic transformation group theory and its applications continually arises 
the problem analogous to (*), but for an action of G on a projective space (in 
fact, this is so in Example 1.9). However, this problem is reduced to Problem 
(*) for actions on vector spaces, see Subsection 2.7. 

1.14. We close this introduction by noting that as 



a constructive solution to Problem (*) provides a constructive solution to the 
following problem: 

How can one find out whether or not two given points of V lie 
in one and the same G- orbit? 

This means that our result yields a constructive solution to the classification 
problem for some types of mathematical objects: for instance, for /c-algebras 
of a fixed dimension up to isomorphisms (see Example 1.7); for ^l-modules of 
a fixed dimension over a fixed fe-algebra A up to isomorphism (see Example 
1.8); for ^-representations of a fixed dimension of a given quiver; for some 
types of algebraic varieties (see Example 1.15). 

Example 1.15. Let /i and /2 be two irreducible forms of the same degree 
in the homogeneous coordinates of the projective space P*^. Assume that, for 
every i = 1,2, the hypersurface Hi in defined by the equation /j = is 
smooth. Let ra ^ 4. Then by a theorem of Scvcry-Lefschetz— Andrcotti every 
positive divisor on the hypersurface Hi is cut out by a hypersurface in P", see 
[MM64, Theorem 2]. It is not difficult to deduce from this that the algebraic 
varieties Hi and H2 are isomorphic if and only if Hi is the image of H2 under 
a projective transformation of P", i.e., if and only if fi is in the GL„+i-orbit 



Remark 1.16. There are the other types of algebraic varieties for which the 
isomorphism problem is reduced to finding out whether or not some forms lie 
in one and the same orbit of the corresponding linear algebraic group. For 
instance, smooth projective curves of a genus g ^ 2 are embedded into P^3-6 
be means of the tripled canonical class, and two curves are isomorphic if and 
only if the image of one of them is transformed to the image of the other by a 
projective transformation of P^^^^. In turn, the latter condition is equivalent 
to the property that the Chow forms (a.k.a. Cayley forms) of these images lie 
in one and the same orbit of the corresponding linear algebraic group. 



2.1. Our further considerations are based on the following fact. 

Lemma 2.2. For some r, s G N, there is a dominant morphism (2). Moreover, 
for r = ikG, there is an open embedding (2) . 

Proof. Let Ru{G) be the unipotcnt radical of the group G. By [R56], as i?u(G) 
is a connected solvable group, the canonical projection G — > G/Ru{G) is a 




of /2. 



2. Main result 
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torsor with the base G/Ru{G) and the structural group Ru{G) that is locally 
trivial in the Zariski topology. By [G58] , as the base is affine and the structural 
group is connected and unipotent, this torsor is trivial. Hence the variety G 
is isomorphic to the product of varieties Ru{G) and G/Ru{G). As Ru{G) is a 
connected unipotent group, the first of them is isomorphic to A*^™ Ru{G) ^ 
[G58]. On the other hand, the big Bruhat cell of the reductive group G/Ru{G) 
is isomophic to K''A^^G/Ru<,G)-r ^ ^Yieie r = TkG/Ru{G) = rkG, see [Spr98]. 
As the variety A''''^''^'^^-'' is isomorphic to A'^'^ ^(^^ x A''''i''"<5/^«(<5)-'', this 
shows that there is its open embedding in G. □ 

2.3. Notation 

Fix a basis ei, . . . , e„ in As the case n = 1 is clear, in the further we 
assume that n > 1. There are functions pij, 1 ^ i, j ^ n, regular on G such 
that the action of G on y is given by the matrix representation 

Pl,l(.9) ••• Pl,n(.9) 



p: G ^Msd.n,nik), p{g) 



Pn,l{g) ■■■ Pn,n{9) 



geG, (3) 



i.e., p{g) is the matrix of the linear V^V,vi-^g-vm the basis ei, . . . , e„, 
so that 

n n n 

g ■ (X] ^i^i) = Yl {Y1 P''^ iahj) Si 5 e G 71, • • • , 7n e A;. (4) 

i=l 1=1 j=l 

Fix a dominant morphism (2): this is possible by Lemma 2.2. Denote by 
Xi, . . . , Xr+s the standard coordinate functions on A'"'*: 

Xi{a) =£i for a = (ei, . . . , Sr+s) € A'"'*. (5) 

As xi, . . . , Xr+s, x]~^, . . . , x~^ generate the /c-algebra /c[A''''*] of regular func- 
tions on A^'* and xi,...,Xr+s are algebraically independent over k, all the 
monomials of the form 

■ ■ ■ x';+'g , h,..., ir+s e Z ir+1, ir+s £ N, (6) 

constitute a basis of the vector space A;[A'''''] over k. 

2.4. The degree of the variety p{G) 

Recall [M76] that the degree of a locally closed subset Y of A' is the car- 
dinality deg Y of the intersection of Y with an {I — dim y)-dimensional linear 
subvariety of A' in general position. For us, the degree deg p{G) of the sub- 
variety p{G) of the space of matrices Mat„.„(/c) is of the special interest. In 
the important case where char /c = and G is a reductive group there is the 
following formula for computation of this number. 

Fix a maximal torus T in G. Let X{T) be its character group in additive 
notation. The latter is a free abelian group of rank r = dimT = rkG naturally 
embedded in the real vector space E = X{T) 0z I^- Fixing a basis in X{T), 
we fix an isomorphism between E and the coordinate space W. Identify these 
spaces by means of this isomorphism. Then the group X{T) is identified with 
the lattice in M''. The Weyl group W := Ng{T) /T naturally acts on E. We 
denote by dv the standard volume form on E. Let Vy be the convex hull in E 
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of the union of zero and the system of weights of the T-module V. Replacing 
G by the quotient group of G by the unity connected component of the kernel 
of the G-action on we may (and shall) assume that this kernel is finite and 
hence dimT^y = r. Fix a system of positive roots of the root system of 
G with respect to T. For every root a G denote by the corresponding 
coroot, i.e., the linear form on E defined by the formula : — > M, (v) = 
2{a\v)/{a\a), where ( | ) is a PF-invariant inner product on E, see [B68]. Let 
mi + 1, . . . , + 1 be the set of degrees of homogeneous free generators of the 
algebra of H-^-invariant polynomial functions on the space E, i.e., mi, . . . ,mr 
are the exponents of W , sec [B68]. 

Theorem 2.5 (Kazarnovskii [Kaz87]). Let char A; = and let p be a represen- 
tation of a connected reductive group G with finite kernel. Then 

Example 2.6. Let char A; = 0. Consider the main object of pre-Hilbertian 
classical invariant theory: G = SL2(A;) and V = Vh is the space of binary 
forms of degree h in variables zi , Z2 over the field k on which G acts by linear 
substitutions of variables: 



g ■ zi = azi +JZ2, g ■ Z2 = Pzi + Sz2, if g 



a (3 
7 5 



G G. (8) 



In this case dimG = 3, \W\ = 2, r = 1, mi = 1, = M, X{T) = 1. 
The set consists of a single root a = 2 and is the standard coordinate 
function on M, i.e., a^(a) = a for every o G M. Take the sequence of monomials 
z^, z^~^Z2, . . . , z\Z2^^,Z2 as a basis ei, . . . , eu+i of V . It follows from (7) that 
Cj+i is a weight vector of the diagonal torus T = {diag(t,t~^) | t G /c \ {0}} 
with weight 1 1— > t^~'^^. Therefore the weight system of the T-module V is the 
arithmetic progression {h,h — 2, . . . , —h + 2, —h}. Hence Vy^ = [—h, h]. The 
kernel of the representation p = ph given by formula (3) is trivial if h odd and 
has order 2 if /i is even. Therefore we deduce from (7) that 



.CT ^ 3! r 2^ \2h^ if /i is odd, 

degpft SL2 = — / x'dx =1 9 

2\hs:iph\J_h \h'^ if /i IS even. 



2.7. The reduction to conic case 

Let L be a finite dimensional vector space over k. Let H be an algebraic 
group (algebraically) acting on a projective space F(iv) of one-dimensional 
linear subspaces of L. Keeping if-orbits in P(-L), we may replace the group H 
by its quotient group by the kernel of action and assume that i7 is a subgroup 
of Aut(P(-L)). Let H be the inverse image of H with respect to the natural 
homomorphism GL(L) — > Aut(P(L)) (note that H is reductive if and only if 
H shares this property). Let vr: L\{0} — > P(-^^) be the natural projection. We 
call a subset in L conic if it is stable with respect to scalar multiplication by 
every nonzero element k. 
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Lemma 2.8. Let U be a nonempty open H-stable subset of P(L) and let 
p,q £ U be two its points. Take any points p G 7r~^(p) and q E Tr~^{q). Then 
the following properties are equivalent: 

(i) the orbit H ■ p lies in the closure of the orbit H ■ q in P(-L); 

(ii) the orbit H ■ p lies in the closure of the orbit H ■ q in U; 

(iii) the orbit H ■ p lies in the closure of the orbit H q in L. 

The orbits H ■ p and H ■ q are conic. 

Proof. As H contains all scalar multiplications of the space L by nonzero 
scalars, H -p = 7r~^{H ■ p) and H -q = vr"^ [H • q) . The reader will easily check 
that the statement follows from this and the definitions. □ 

We shall need the following application of Lemma 2.8. Let L be the coordi- 
nate space and let H be the group G from Subsection 1.1 that acts on 
P(L) according to the rule (see (3)) 



9- 



(n n \ 

ao ■ '^Pi,i{9)oii ■ ■ ■ ■ ■ '^Pn,i{9)ai j ■ 
i=l i=l ' 



The standard principal open subset {(ao : a\ : . . . : | ao / 0} of P(-L) 
is G-stable and is equivariantly isomorphic to the G-module V . Hence, by 
Lemma 2.8, answering question (*) is equivalent to answering whether or not 
the orbit G ■ a lies in the closure of the orbit G ■ b. This means that replacing 
the group G by G, the space V by L, and the points a and b by, respectively, 
a and b, we reduce solving Problem (*) to the case where both orbits G ■ a and 
G ■ b are nonzero and conic. Given this, 

searching for an answer to Problem (*), we may assume that 

G ■ a and G ■ b are nonzero conic orbits. ^ ^ 

Note also that by Lemma 2.8 the problem analogous to Problem (*), but 
for an action on a projective space is reduced to Problem (*) for an action on 
a linear space. 

2.9. The input of the algorithm 

We assume that the following data are known (cf. [Po81]): 

— The degree of the variety p{G), 

d:=degp{G). (11) 

— The functions 

i-* {Pp,q) ^k[A'''''] = k[xi, . . .,Xr+s,Xi^, ■ ■ ■,x~'^], I ^p,q^n. 

Example 2.10. Consider the same situation as in Example 2.6. Number (11) 
is given by formula (9). It follows from (8) that the functions pp^q in (3) are 
defined by the equality 



{az,+^Z2)''-^{Pz, + dz2y = J2pi+wi9)zt'4, 5= " ^ 



eG. (12) 
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Take l to be the morphism 



i: 



.1,2 



SL2(A;), 



(£l,£2,^3) 



"1 


£2 




£1 " 




1 


o" 







1 









.^3 


1 





^1 ^3 ^1 



(13) 



Then it follows from (5), (12), (13) that the function i*(pj+ij+i) is equal to 
the coefficient z^'^z^ in the decomposition of the binary form 

((Xl + X^^X2X3)zi + {X^^X3)Z2)^ \{x^^X2)zi + {x^^)z2y 

in the variables zi,Z2 with the coefficients in the field k(xi,X2,x-i) as sum of 
monomials in zi, Z2. For instance, if /i = 2, then t*(p2,2) = 1 + 2xj^^X2X3. 

2.11. The algorithm 

Now we turn to the formulation and proof of the main result. We utilize 
the notation and conventions introduced above and exclude the trivial case 
G b = V, i.e., assume that 



dim G ■ b < dim V 



(14) 



(as the number dim G • 6 is equal to the rank of the system of vectors {dp{Yi) ■ 
b}i^i where {iijie/ is a basis of the vector space Lie(G), condition (14) can 
be verified constructively if the operators dp{Yi) arc known). 

The following sequence of steps together with Theorem 2.12 provide a const- 
ructive method to answer Problem (*): 

(1) Find the coordinates of the vectors a and b in the basis ei, . . . , e„: 

a = aiei + . . . + UnCn, b = (5iei + . . . + /?„e„, 

and, changing the basis ei, . . . , e„ if necessary, achieve that the following 
condition holds: 



^•••/3n7^0. 



(15) 



(2) Consider n "generic" polynomials Fi, . . . , Fn of degree 2d — 2 (where d 
is defined by formula (11)) in the variables yi, . . . , y„. 



p ■- 



E 



,91 



P 



,n, 



(16) 



(Ji,...,(J„GN 
?l+-+9n^2d-2 



i.e., such that, apart from yi, . . . , yn, all the coefficients Cp^qj^...^q„ are the 
indeterminates over k as well, and put 



Hivi, . . . ,y„) := (yi - ai)Fi + . . . + (yn - otn)Fn - 1. 



(17) 



(3) Replacing in H{yi, ... ,yn) every variable yi by Yl]=i f^j^*iPi,j)^ obtain 
a linear combination of monomials of form (6) with the coefficients in 
the ring k[. . . , Cp^qi,...,q„, . . .] of polynomials in variables Cp^q^^„_^q^ over the 
field /c: 
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(n n 

7=1 7=1 



{Pn,j 



{ii,...,ir+s)eM 

where ^ii,...,^,,^^ k[. . . , Cp^q^^,,,^q^, . . .] and M is a finite subset in Z'' x N*. 
By (16) and (17), every coefficient in (18) is a linear function 

in the variables Cp^g^,,.,,g„ with the coefficients in the field A;. 

(4) Consider the following finite system of linear equations in the variables 
Cp,qi,...,q„ with the coefficients in the field k: 

^h,...,ir+s = 0' where (zi , . . . , v+s) e M. (19) 

Theorem 2.12. Let G ■ b be a nonzero conic orbit {see (10)). The following 

properties are equivalent: 

(i) the closure of the orbit G ■ b inV contains the orbit G ■ a; 

(ii) system of linear equations (19) is inconsistent. 

Proof. We split it into several steps. 

1. Let zi,. . . ,Zn be the basis of V* dual to ei, . . . , Cn. Let ti be the re- 
striction to G • 6 of the function Zi. As the set G • 6 is closed in V and 
k[V] = k[zi, . . . , Zn], we have 



k[G-b] = k[tu...,tn]. (20) 

Note that the function ti is not a constant. Indeed, as the orbit G • 6 is 
conic, G G • 6. The definition of ti implies that ti{0) = and ti{b) = Pi. But 
Pi ^ because of (15). 

Consider the orbit morphism 



ip-.G^G-b, ip{g)=g-b. (21) 
As the morphism l is dominant, the image of the morphism 



^■.= <poL: K^^' ^G-b (22) 



is dense in G • 6; whence the corresponding comorphism is an embedding of 
the algebra of regular functions: 



^l;*:k[G-b]^k[K''^']. (23) 

It follows from (20) that 



r{k[G-b]) = k[r{ti),...,r{tn)], (24) 

and (21), (22), (4), and the definition of ti imply that 

n 

r{u) = Y.Pji*{Pi,j). (25) 
j=i 

There is only one point of V where cill the fnnctions z\ — Q^i; • • • 5 ~^ 
vanish, the point a. Taking into account that the set G - h is G-stable, we 
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deduce from this that the following properties are equivalent: 

(ci) the orbit G ■ a does not lie in the closure of the orbit G ■ b; ' 
(C2) the point a does not lie in the closure of the orbit G ■ b; 
(ca) there are no points of G • 6 where all the functions 1 
ti — cti, . . . ,tn — ctn vanish. 

2. Now we shall use the effective form of Hilbert's NuUtellensatz obtained 
in [J05]. In order to formulate this result we shall introduce some notation 
and definitions. 

First, for every positive integers di ^ ■ ■ ■ ^ dm and q, set 



N{di,...,dm;q) = < 



YY^i di if g ^ m ^ 1, 

(nf=i di)dm if m > g > 1, 
dm if 9 = 1, 

and also 

N'{di,...,dm;q) = N{di,...,d„uq)\iq>l, and N' {di, . . . ,dm.-A) = di. 

Further, if a nonzero regular function h on an irreducible closed subset X 
of an affine space A' is given, the minimum of degrees of polynomial functions 
on a' whose restriction to X is /i will be called the degree of h and denoted 
by deg/i. It is easily seen that, for every nonzero regular functions / and h 
on X, the inequality degfh ^ deg/ + degh holds and, in general, it may be 
strict. However, if the set X is conic, then necessarily 

deg//i = deg/ + deg/i. (27) 

Theorem 2.13 (Z. Jelonek [J05]). Let X be an irreducible closed subset of 
of positive dimension. Let hi, ... , hm be the nonconstant regular functions on 
X such that 

deg/ii ^ • • • ^ deghm- (28) 
Then the following conditions are equivalent: 

(a) there are no points of X where all the functions hi, ... , hm vanish; 

(b) there are regular functions fi, . . . , fm on X such that 

m 

l = ^fihi 

i=l 

and, for every i, the following inequality holds: 

J ^ i, ^ jdegX-A/"'(deg/ii,...,deg^^;dimX), «/ m ^ dim X, 

deg fihi ^ < 

I 2degX-A/" (deg/ii, . . . ,deg^^;dim X) — 1, if m > dim X. 



Now we take as A' and X respectively V and G ■ b. As the nonconstant 
function ti is the restriction to G • 6 of a linear function on F, we have 

deg (ti-ai) = ... = deg (*„ - «„) = 1. (29) 

Further, in Theorem 2.13 put m = n and hi = ti — ai, i = 1, . . . ,n (by (29) 
condition (28) is fulfilled). Then it follows from (26), Theorem 2.13, and (14), 
(29) that every property (ci), (C2), (cs) in (26) is equivalent to the property 
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(C4) there are functions fi, ■ ■ ■ , fn regular on G ■ b such that 

YliLiiU — oii)Ji — 1 = and, for every i, the following inequality holds: 

deg(ti-ai)/i ^2degG^-l. (30) 

As G • 6 is a conic (irreducible) subvariety of V ^ it follows from (27) and 
(29) that deg (tj — ai)fi = 1 + deg/j. Therefore inequality (30) is equivalent 
to the inequality 

deg/i ^2degG^-2. (31) 

3. The degree of the variety G ■ h can be upper bounded. Namely, the 
orbit G • 6 is the image of the variety p{G) C End(y) under the linear map 
End(T^) -^V, g-h. But it is easy to prove (see, e.g., [DK02, Prop. 4.7.10]) 
that degree does not increase under afline maps: if y is a locally closed subset 
of A' and : A' ^ A™ is an affine map, then degY ^ dQgip(Y). Therefore 
degG • h is not bigger than the degree of the subvariety p{G) in End(F), i.e., 
the number d. Hence, by virtue of (31), for every i, the following inequality 
holds 

deg/i^2d-2. (32) 

4. As the comorphism ip* (see (23)) is an embedding, we obtain the equiva- 
lence 

m m 

5Z(ti - ai)/i - 1 = ^ ^irik) - ai)r{fi) - I = 0. (33) 
1=1 i=l 

By virtue of (25) and (33), it follows from inequality (32) and the definitions 
of functions ti, numbers deg/j, the "generic" polynomials Fp (see (16)), and 
the polynomial H (see (17)) that property (04) is equivalent to the following 
property: 

(C5) for every coefficient Cp^q^^,,,^q^ of every "generic" polynomial Fp, there 
is a constant G k such that after substitution of fp,gi,...,g„ in 

place of Cp^q^^...^q„ for every p,qi, . . . , qn, the right-hand side of formula 
(19) becomes zero of the field of rational functions in xi, . . . , Xr+s with 
the coefficients in k: 

^n,...,v+s(- ■ ■ ) typ^gi,...,q„, ■ ■ ■)xY ■ ■ ■ x^^g = 0. (34) 

(n,...,v+s)eM 

It remains to notice that as monomials x^-^ • • • a;*'^^', where (ii, . . . , ir+s) G 
M, are linearly independent over k, equality (34) is equivalent to vanishing of 
all the coefficients of the left-hand side, 

i.e., to that Cp^q-^^,_,^q^ = i^p^q^^^.^q^ IS & solution of system of linear equations 
(19) in variables Cp,5i,...,q„. This completes the proof of the theorem. □ 

Remark 2.14. The proof shows that the claim of Theorem 2.12 remains true 
if the constant d in the definition of the "generic" polynomials Fi,. . . ,Fn is 
replaced by deg G ■h = deg G ■ b. If, from some reasons, the number deg G ■ 6 is 
known, this permits to decrease the number of variables and equations in the 
system of linear equations (19). In some cases the degrees of orbits indeed 
have been computed. 
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Example 2.15. Consider the same situation as in Examples 2.6 and 2.10. 
Take a nonzero binary form v £ and decompose it as a product v = 
v^^ ■ ■ ■ Vp'' , where vi, . . . ,Vp are pairwize nonproportional forms from Vi. Assu- 
me that p ^ 3 and /i/nj ^ 2 for every i. Then the G-stabiHzer Gy of the form v 
is finite [Po74] and \G^\degG-i) = -2{p-l)h^ -A^P^^{h-nif+3h'^ Ef=i(^- 
ni) + 3hY^^^i{h — ni){h — 2ni) (see the proof in [MJ92, Sect. 8]). In particular, 
if all the roots of the form v are simple, i.e., p = h, rii = . . . = rih = 1, then 

|G^|deg(G-t;) = 2/i(/i - l)(/i - 2). (35) 

Formula (35) can also be deduced from a calculation made in 1897 by Enriques 
and Fano; this has been done in 1983 by Mukai and Umemura (with a gap 
fixed in [MJ92, Sect. 8, Remark] where one can find the relevant references). 



3. Defining the set G • L hy equations 

3.1. Let L be a linear subvariety of V. Then there is a morphism 

r: A' V, 

whose image is dense in L: for instance, one can take r to be an afHne embed- 
ding of a' into V whose image is L. We fix such a morphism r. Besides, like 
above we assume that a dominant morphism (2) is fixed. 

We maintain the notation from Subsection 2.3. Like in the proof of Theo- 
rem 2.12, we denote by . . . , Zn the basis of V* dual to ei, . . . , e„. Besides, 
we denote by yi, . . . ,y; the standard coordinate functions on A': 

yi{a) = 6i for a = {Si, ...,6i)e A'. 

Then 

n 

t{v) = ''^^T*{zi){v)ei for every G a'. (36) 
1=1 

3.2. The functions xi, . . . ,Xr+sj Vi^ ■ ■ ■ iVl can be naturally extended to the 
functions on A'"'* x A'; we denote these extensions by the same letters. Consider 
the morphism 

^ : A'''' xA^^V, n{u, v) = i{u) ■ t{v) (37) 
Then (4) and (36) imply that 

n 

fp := H*{zp) = ^L*{ppq)T*{zq), l^p,q^n. (38) 

q=l 

We identify A^''' X A' with the open subset of A''"^*''"' by means of the 
embedding 

A'^'" X A^ A''+"+', {{ei...,£r+,),i5i,...,6i)) ^ (ei . . . , e^+„ 5i, . . . , 5;); 

then xi,...,Xr+s, yi,---,yi become the standard coordinate functions on 
^r+s+l_ Besides, we identify V with A" by means of the isomorphism 

n 

F ^ A", ^ 'jid (71, . . . , 7„). 

i=l 
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Then morphism (37) becomes the rational map q of the affine space A''"^*+' to 
the affine space A": 

g; K'-+s+l A", a ^ (/i(a), . . . , /„(a)). 



As t(A^'*) ■ L = G ■ L, we have the equahty 



g(^^r+s+i^ = GL. (39) 

This makes it possible to apply eliminaton theory to finding the equations 
that cut out G ■ L in V. An algorithmic solution to this problem is obtained 
by means of Grobncr bases as follows. 

3.3. The input of the algorithm 

We assume that the following data are known: 

— The functions 

i*(pp,g)Gfe[xi,...,x,.+3,xJ"\...,a;7^] C fc(A''+^+'), l^p,q^n. (40) 

— The functions 

T*{zi) € k[yu . . . ,yi] C k[A^+'+% l^i^n. (41) 

Example 3.4. Fix a point v ^ L and a sequence /i, . . . , /m of linear indepen- 
dent vectors defining a parametric presentation L = {v+Y^"!^i Xifi \ Xd, ■ ■ ■ , 
G A;}. Take r to be the embedding r: A"* ^ A", t{Xi, . . . , Xm) = v + 
El=m ^ifi- Let V = YTj=i Ijf^j and U = Y]}=i ^ji^j- Then 

n 

1=1 

See Example 2.10 regarding the functions i*{pp^q). 

3.5. The algorithm 

The following sequence of steps together with Theorem 3.6 provide a const- 
ructive method to obtain the equations defining G • L in T^: 

(1) Compute the rational functions fp using formula (38) and write down 
each of them fraction of polynomials: 

fp — T~' 1 9p ^ ■ ■ ■ J ^r+s ) 2/1 ) • • • ) 2/t] ) hp & k [xi , . . . ,Xr] 

tip 

(see (40) and (41)). 

(2) Consider the polynomial ring k[t, xi, . . . , Xr+s, Ui, ■ ■ ■ ,yt, zi, . . . , Zn], whe- 
re t is a new variable, and find for its ideal generated by the polynomials 

^1^1 - gi,---, hnZn - gnA - hi ■ ■ ■ hnt. 

a Grobncr basis with respect to an order of monomials such that every 
variable t, xi yj is bigger than every variable Zp. 

Theorem 3.6. Let qi,...,qm be all the elements of this Grobner basis that 
lie in k[zi, . . . ,Zn]- Then 



G-L = {veA^\qiiv) = ... = qm{v) = 0}. 
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Proof. We have ^(A''+*+') = {u G A" | qi{v) = ... = qm (v) = 0} — this is a general 
fact about the closure of image of every rational map of one affine space to 
another, see, e.g., [CL098, Chap. 3, §3, Theorem 2]. Now the claim that we 
wish to prove follows from equality (39). □ 

Remark 3.7. Although the elements qi, . . . , qm, interesting for us, constitute 
a part of the Grobner basis, for finding them by means of the described algo- 
rithm, we have to find the whole of this basis. 
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